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^5 WHAT IS CLAIMED IS : 

1 . A method for tracking the position and orientation of an object, comprising the steps of: 

(a) scanning across an object to detect fiducials, wherein a video run is formed by a scan; 

(b) clumping video runs to detect a pattern of fiducials; 

(c) acquiring estimated values for a set of tracking parameters by comparing a detected 
1 0 pattern of fiducials to a reference pattern of fiducials; and 

(d) iterating the estimated values for the set of tracking parameters until the detected 
pattern of fiducials match the reference pattern of fiducials to within a desired convergence. 

2. The method for tracking the position and orientation of an object of claim 1, wherein the step 
15 of scanning across an object to detect a fiducials includes the step of setting a predetermined 

threshold voltage level for detecting a fiducial, and identifying fiducial edges when an output 
voltage from an optical sensor crosses the predetermined voltage level. 

3. The method for tracking the position and orientation of an object of claim 1, wherein the step 
20 of clumping video runs includes the step of combining adjacent video runs and extracting 

relevant information from the video runs. 

4. The method for tracking the position and orientation of an object of claim 3, wherein a pixel 
is recorded by recording a scan line number and a pixel number for each pixel that has a video 

25 level above the predetermined threshold. 

5. The method for tracking the position and orientation of an object of claim 1, wherein the step 
of clumping video runs includes the steps of detecting and removing noise from the video runs. 

30 6. The method for tracking the position and orientation of an object of claim 1 , wherein the 
detected and reference patterns of fiducials includes a geometrically consistent pattern of hard 
fiducials and wherein the step of acquiring estimated values for a set of tracking parameters 
includes the step of corresponding a predetermined number of detected hard fiducials with the 
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"5 reference pattern of fiducials to estimate phi, theta and psi orientation parameters and to estimate 
a distance position parameter. 

7. The method for tracking the position and orientation of an object of claim 1, wherein the 
detected and reference patterns of fiducials includes a pseudo random pattern of soft fiducials 

10 and wherein the step of acquiring estimated values for a set of tracking parameters includes the 
step of electing at least one of the soft fiducials with the reference pattern of fiducials to estimate 
the X-bar and Y-bar position parameters. 

8. The method for tracking the position and orientation of an object of claim 1, wherein the step 
1 5 of acquiring estimated values for a set of tracking parameters includes a cold acquire step and a 

quick acquire step, wherein the quick acquire step uses prior tracking parameters to estimate new 
tracking parameters, and wherein the cold acquire step is performed when prior tracking 
parameters are not available to estimate new tracking parameters. 

20 9. The method for tracking the position and orientation of an object of claim 1 , wherein the step 
of iterating the estimated values for the set of tracking parameters uses the method of least 
squares. 

10. The method for tracking the position and orientation of an object of claim 1 , further 
25 comprising the step of performing a startup routine, said startup routine including the steps of 
loading real world alignment data from a computer database file, calculating compensation for 
know aberrations of a sensor system used in the step of scanning across an object to detect 
fiducials, and setting a gain for the sensor system. 

30 1 1 . A method for augmenting reality, comprising steps of: 

(a) tracking the position and orientation of a pattern of fiducials on an object with a self- 
contained, mobile system; 

(b) processing virtual information stored in a computer memory of said system according 
to the position and orientation of the object; and 
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5 (c) presenting the virtual information with real information to a user in near real time 

with said system. 

12. The method for augmenting reality of claim 11, wherein the pattern of fiducials are disposed 
on the object surface in a geometrically consistent hard fiducial pattern and in a pseudo random 

10 soft fiducial pattern. 

13. The method for augmenting reality of claim 11, wherein said step of tracking the position 
and orientation of the object includes the steps of: 

scanning across the object to detect the fiducials, wherein a video run is formed by a 

1 5 scan; 

clumping video runs to detect the pattern of fiducials; 

acquiring estimated values for a set of tracking parameters by comparing a detected 
pattern of fiducials to a reference pattern of fiducials; and 

iterating the estimated values for the set of tracking parameters until the detected pattern 
20 of fiducials match the reference pattern of fiducials to within a desired convergence. 

14. The method for augmenting reality of claim 13, wherein the said step of scanning across the 
object to detect the fiducials includes the steps of setting a predetermined threshold voltage level 
for detecting a fiducial and identifying fiducial edges when an output voltage from an optical 

25 sensor crosses the predetermined voltage level. 

15. The method for augmenting reality of claim 13, wherein said step of scanning across the 
object to detect the fiducials includes the step of forming a signal corresponding to the detected 
pattern of fiducials, removing noise from the signal, and reducing the bandwidth of the signal. 

30 

16. The method for augmenting reality of claim 13, wherein said step of acquiring estimated 
values for a set of tracking parameters includes the steps of: 
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5 corresponding a predetermined number of detected hard fiducials with the reference 

pattern of fiducials to estimate phi, theta and psi orientation parameters and to estimate a distance 
position parameter; and 

electing at least one of the soft fiducials with the reference pattern of fiducials to estimate 
X-bar and Y-bar position parameters. 

10 

17. The method for augmenting reality of claim 11, wherein the object surface includes a wiring 
board used in a process of fabricating wire harnesses, the virtual information includes a wiring 
schematic and instructions for a wiring harness, and the real information includes the wiring 
board, wires and connectors. 

15 

18. The method for augmenting reality of claim 11, wherein the computer memory forms a part 
of a wearable computer, the wearable computer having processor which performs said step of 
processing virtual information stored in the computer memory according to position and 
orientation of the object. 

20 

19. The method for augmenting reality of claim 11, wherein said step of presenting the virtual 
information with real information to a user in near real time includes the step of projecting the 
virtual information on a head mounted display. 

25 20. The method for augmenting reality of claim 1 1 , wherein said step of presenting the virtual 
information with real information to a user in near real time includes the step of projecting the 
virtual information on an Optical See Through display. 

2 1 . The method for augmenting reality of claim 1 1 , wherein said step of presenting the virtual 
30 information with real information to a user in near real time includes the step of projecting the 

virtual information and the real information on a Video See Through display. 

22. The method for augmenting reality of claim 11, wherein said steps of tracking the position 
and orientation of the object, retrieving virtual information, and presenting the virtual 
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5 information with real information are performed with an update rate of at least 60 Hertz and a 
latency below 16 milliseconds. 

23. The method for augmenting reality of claim 11, wherein the virtual information includes 
menu driven screen displays, wherein the screen displays forming a user-friendly interface for an 

1 0 augmented reality system, end wherein a user selects menu item by moving a cursor over a 
desired selection and choosing the selection. 

24. The method for augmenting reality of claim 23, wherein the cursor is moved by moving the 
user's head. 

15 

25. The method for augmenting reality of claim 1 1 , further comprising the step of calibrating the 
alignment between the virtual information and the real information. 

26. A method for augmenting reality, comprising steps of: 

20 (a) using a sensor to provide at least one signal that is indicative of a pattern of fiducials 

on an object; 

(b) processing said signal to locate said fiducials; 

(c) determining a relative position and orientation of the sensor with respect to the object 
by comparing the locations of said fiducials to a known reference pattern; 

25 (d) providing virtual information to a user in substantial registration with real 

information based on the relative position and orientation determined in step (c), and wherein 
said method is performed so as to provide said virtual information to said user in near real-time. 

27. A method according to claim 26, wherein step (d) comprises providing computer-generated 
30 aural information to a user. 

28. A method according to claim 26, wherein step (d) comprises providing computer- generated 
kinesthetic information to a user. 
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5 29. A method according to claim 26, wherein step (a) comprises using a sensor to provide at 
least one signal that is indicative of a pattern of fiducials on a user's eye. 

30. A method according to claim 26, wherein said sensor is mounted on a user's head. 

10 3 1 . A method according to claim 26, wherein steps (a) through (d) are performed with a self- 
contained, mobile system that comprises a wearable computer. 

32. A method according to claim 26, wherein at least one of steps (b) and (c) are performed by 
electronically reducing said virtual information to a reduced amount of information for 
1 5 manageable processing. 



33. A method according to claim 32, wherein said step of electronically reducing said virtual 
information comprises electronically filtering said information to eliminate background 
information. 

34. A method according to claim 32, wherein said step of electronically reducing said virtual 
information comprises electronically filtering said information to eliminate noise. 



35. A method according to claim 34, wherein said step of electronically filtering said 
25 information to eliminate noise comprises electronically correcting to compensate for known 
aberrations of at least one hardware element. 



36. A method according to claim 32, wherein said step of electronically reducing said virtual 
information comprises removing at least one synchronization signal from said information. 

37. A method according to claim 32, wherein said step of electronically reducing said virtual 
information comprises a step of determining a centroid position of at least one of said fiducials. 
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"5 38. A method according to claim 37, wherein said step of electronically reducing said virtual 
information further comprises utilizing said centroid position in subsequent electronic 
processing. 

39. A method according to claim 37, wherein said step of determining a centroid position of at 
1 0 least one of said fiducials comprises steps of determining at least two boundary locations of a 

fiducial and calculating the centroid position based at least in part on said boundary locations. 

40. A method according to claim 37, wherein said step of determining a centroid position of at 
least one of said fiducials comprises detecting a beginning pixel and an ending pixel of the 

1 5 fiducial during a video raster run; extracting the line number, the beginning pixel number and the 
number of continuous pixels after the beginning pixel; processing such information obtained 
from different line numbers of the video raster run to form individual fiducial representations; 
and further processing this information to determine the centroid position. 

20 4 1 . A method according to claim 26, wherein step (a) comprises tracking the position and 
orientation of at least one hard fiducial pattern having a known geometric shape and further 
comprises tracking the position and orientation of at least one soft fiducial pattern. 

42. A method according to claim 41, wherein at least one fiducial of said soft fiducial pattern is 
25 located within said known geometric shape. 

43. A method according to claim 41, wherein step (b) comprises a step of locating said hard 
fiducial pattern, and wherein said step of locating said hard fiducial pattern is aided by utilizing a 
reference pattern of fiducials corresponding to said known geometric shape. 

30 

44. A method according to claim 41, wherein step (b) further comprises locating said soft 
fiducial pattern, and wherein said step of locating said soft fiducial pattern is aided by said prior 
determination of said hard fiducial pattern. 
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"5 45. A method according to claim 41, wherein step (b) comprises electronically calculating the 
position and orientation of the object based on the determined location of the hard fiducial 
pattern and the determined location of the soft fiducial pattern. 

46. A method according to claim 1 12, wherein said step of electronically calculating the position 
10 and orientation of the object is performed based on a relative position of at least one fiducial 

within said soft fiducial pattern that is located within said known geometric shape of said hard 
fiducial pattern. 

47. A method according to claim 26, wherein step (a) comprises iteratively tracking the position 
1 5 and orientation of a number of fiducials on said object. 

48. A method according to claim 47, wherein said step of iteratively tracking the position and 
orientation of a number of fiducials comprises a quick acquire routine for calculating a 
theoretical parameter value for at least one fiducial based on a previous position of the fiducial. 

20 

49. A method according to claim 48, wherein said quick acquire routine tracks movement of the 
system with respect to the object and utilizes data pertaining to said movement to predict likely 
future locations of said fiducial. 

25 50. A method according to claim 47, wherein at least one of steps (b) and (c) comprises receiving 
a signal of information based on said iterative tracking and processing said signal to eliminate 
nonessential data, thereby reducing bandwidth. 
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